The  Center  for  Army  Acquisition  Lessons  Learned  (CAALL)  was  es¬ 
tablished  within  the  U.S.  Army  Materiel  Systems  Analysis  Activity 
(AMSAA)  to  collect,  analyze  and  disseminate  acquisition  lessons 
learned.  The  center  serves  as  the  authoritative  source  for  timely, 
real-world  acquisition  lessons  learned  to  enhance  the  performance 
of  the  Army's  project  offices  in  support  of  the  warfighter. 

The  mission  originated  with  the  2010  Army  Acquisition  Review,  which  repeatedly  cited 
the  need  for  a  centralized  source  for  lessons  learned  across  the  acquisition  life  cycle.  The 
report  stated  that  there  are  lessons  learned  within  the  Office  of  the  Assistant  Secretary  of 
the  Army  for  Acquisition,  Logistics  and  Technology  (ASA[ALT]),  the  test  community  and 
other  acquisition  stakeholders,  but  they  are  all  dispersed  and  need  to  be  synthesized.  In 
addition,  the  report  stated  there  is  no  formal  way  to  track  successes,  analyze  failures  and 
develop  best  practices  from  historical  programs.  Therefore,  the  acquisition  community 
needs  a  robust,  readily  accessible  database  and  associated  analytical  capability  to  extract 
relevant  information.  The  report  recommended  that  a  Center  for  Army  Acquisition  Lessons 
Learned  be  established  to  provide  a  record  of  acquisition  experiences  in  order  to  allow 
others  within  the  acquisition  community  to  understand  what  occurred  and  avoid  previous 
mistakes,  as  well  as  to  provide  the  basis  for  making  improvements. 


Consequently,  the  Army  Acquisition  Executive  (AAE)  distributed  a  memorandum  in  Janu¬ 
ary  2012  directing  all  Army  acquisition  programs,  regardless  of  Acquisition  Category 
(ACAT),  to  conduct  After  Action  Reviews  (AARs)  and  document  lessons  learned  following 


Iracki  is  an  operations  research  analyst  on  the  Acquisition  Lessons  Learned  Team,  Acquisition  Studies  and 
Analysis  Branch,  U.S.  Army  Materiel  Systems  Analysis  Activity  at  Aberdeen  Proving  Ground  in  Maryland. 
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all  milestone  reviews  and  program  terminations.  The  memo¬ 
randum  also  directed  AMSAA  to  establish  CAALL  to  collect 
these  lessons  via  a  Web-enabled  database  and  to  conduct 
analysis  on  the  lessons  learned. 

Acquisition  Lessons  Learned  Portal  (ALLP) 
and  Lessons  Learned  Collection 

CAALL  has  established  the  ALLP  as  the  authoritative  source 
for  Army  acquisition  lessons  learned.  The  ALLP  aims  to  serve 
as  a  knowledge  management  tool  for  the  program  executive 
offices  (PEOs)  and  their  project  offices,  as  well  as  the  broader 
acquisition  community.  The  primary  function  of  the  portal  Is  to 
allow  easy  input  and  retrieval  of  lessons  learned.  To  facilitate 


key  words  and  ACAT  level  to  which  the  lessons  apply.  This  in¬ 
formation  aids  users  in  finding  lessons  that  may  be  applicable 
to  their  programs  or  types  of  work.  It  also  is  critical  to  CAALL's 
analysis  processes,  as  It  allows  the  team  to  look  at  frequently 
used  categories  and  key  words  to  help  identify  trends  within 
the  lesson-learned  submissions. 

Lesson  Searches 

The  ALLP  includes  a  lessons-learned  search  page,  where  users 
may  easily  find  lessons  pertaining  to  their  interests  through  a 
text-based  keyword  search.  Users  may  refine  their  search  cri¬ 
teria  using  filters  for  the  category,  ACAT,  milestone  and  phase 
to  which  the  lesson  applies.  The  search  will  return  a  table  of 
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collection  of  acquisition  lessons  learned,  the  ALLP  provides 
an  online  form  for  lesson  submission,  as  well  as  a  download¬ 
able  form  that  can  be  completed  offline  and  then  uploaded 
to  the  portal.  This  allows  the  form  to  be  filled  out  and  distrib¬ 
uted  through  different  offices  for  review  prior  to  submission 
if  needed. 

Lesson  Input 

The  main  fields  on  the  form  include  the  Lesson  Learned,  Back¬ 
ground,  Recommendation,  and  Cost,  Schedule  and  Perfor¬ 
mance  Impacts.  The  Lesson  Learned  field  is  a  concise  (maxi¬ 
mum  of  200  characters),  specific  and  actionable  statement 
that  describes  the  knowledge  the  author  gained  through  the 
experience  that  can  benefit  other  programs  if  shared  and  reap¬ 
plied.  This  allows  readers  to  easily  and  quickly  identify  the  les¬ 
son  and  determine  whether  they  would  like  to  read  further  in 
the  Background  and  Recommendation  fields.  The  Background 
for  the  lesson  describes  the  events  observed  or  the  actions 
taken  and  why  they  were  taken.  The  Recommendation  field 
provides  details  on  how  the  lesson  can  be  reapplied  in  the 
future  and  how  it  can  benefit  other  programs  or  organizations. 
The  form  also  has  fields  for  capturing  impacts  to  the  program's 
cost,  schedule  and  performance.  This  information  allows  users 
to  get  an  idea  of  the  possible  impact  of  the  recommendation 
on  another  program  if  reapplied  and  allows  CAALL  analysts  to 
identify  those  issues  having  the  largest  impacts  on  programs. 
In  addition,  the  form  collects  metadata  for  the  lessons,  such  as 
phases  and  milestones  of  the  acquisition  life  cycle,  categories. 


lessons  along  with  any  other  Information/fields  the  user  speci¬ 
fies  should  be  included  in  the  search  results  table.  Users  may 
click  on  the  Lesson  Learned  text  for  each  lesson  that  appears 
in  the  search  results  to  view  the  full  lesson-learned  record. 

Collaborative  Tools 

The  portal  includes  other  collaborative  tools,  such  as  a  docu¬ 
ment  repository  and  user  forums.  In  the  document  reposi¬ 
tory,  users  may  share  detailed  documents  pertaining  to  their 
lessons  learned,  such  as  lessons-learned  reports,  useful 
templates,  guidance,  etc.  In  the  user  forums,  users  may  post 
questions  and  informally  discuss  acquisition  issues  with  other 
members  of  the  acquisition  workforce. 

Spotlight  Zones 

The  ALLP  includes  two  Spotlight  Zones— Web  pages  focused 
on  a  particular  hot  topic  in  acquisition.  The  Spotlight  Zones  aim 
to  provide  the  acquisition  community  with  specific  Informa¬ 
tion  that  will  aid  programs  in  those  areas  of  acquisition  that 
are  receiving  significant  attention  in  the  acquisition  world.  The 
current  Spotlight  Zones  include  Reliability  and  Modeling  and 
Simulation  and  provide  lessons  learned,  case  studies,  links, 
guidance  and  tools  pertaining  to  these  two  topics. 

Populating  the  Portal  with  Lessons 

Upon  establishment  of  the  ALLP,  CAALL  began  to  populate  the 
acquisition  lessons-learned  database  with  historical  lessons 
from  existing  sources.  The  team  gleaned  lessons  from  various 
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reports,  such  as  the  RAND  Corp/s  report  on  lessons  from  the 
Future  Combat  Systems  as  well  as  Government  Accountability 
Office  reports.  CAALL  has  had  continuous  interactions  with 
the  PEOs,  as  well  as  the  greater  acquisition  community,  to 
solicit  lessons  learned  based  on  their  real-world  experiences. 
CAALL  regularly  attends  Army  Systems  Acquisition  Review 
Council  meetings  to  stay  informed  on  programmatic  deci¬ 
sions  and  gather  lessons  or  potential  topics  for  lessons  to  be 
developed  by  the  project  manager  (PM).  In  addition,  PEOs 
and  project  offices  have  begun  to  populate  the  database  with 
lessons  from  AARs  following  milestone  reviews  as  directed  by 
the  ASA(ALT)  memorandum,  as  well  as  informal,  unprompted 
lesson  submissions.  The  ALLP  currently  has  over  600  users 
and  provides  access  to  more  than  500  lessons  learned. 

Analysis  and  Dissemination 

A  key  element  of  the  acquisition  lessons-learned  mission  is 
the  analysis  of  lesson-learned  submissions.  This  includes 
trend  analysis  of  lesson  submissions  and  deep-dive  analy¬ 
ses  of  specific  topics  or  trends,  as  well  as  case  studies  of 
particular  Army  acquisition  programs.  The  AAE  has  rec¬ 
ognized  the  need  to  continually  identify  the  top  five  issues 
affecting  Army  programs  and  the  need  to  have  data  to  sup¬ 
port  these  findings.  CAALL  synthesizes  lessons  and  identi¬ 
fies  trends  to  provide  to  ASA(ALT)  in  an  effort  to  address 
systemic  challenges  and  provide  the  basis  for  acquisition 
policy  changes  and  strategic  decisions.  CAALL  has  begun 
to  conduct  deep-dive  analyses  of  particular  acquisition  is¬ 
sues  that  have  emerged  from  the  synthesized  trends.  One 
such  trend  that  has  been  identified  Is  that  documentation 
preparation  and  approval  processes  are  resource  drains  for 
project  offices.  Consequently,  CAALL  is  conducting  a  deep¬ 
dive  study  on  acquisition  documentation  requirements  and 
staffing  in  an  effort  to  determine  which  documents  cause 
the  most  Issues,  where  duplication  exists  in  documenta¬ 
tion  requirements,  and  where  there  are  inefficiencies  within 
the  staffing  and  approval  processes.  Analyses  such  as  this 
will  be  presented  to  ASA(ALT)  leadership  to  provide  de¬ 
tailed  findings  and  potential  recommendations  for  process 
changes.  Furthermore,  CAALL  conducts  case  studies  on 
programs  that  have  had  significant  learning  experiences 
that  led  to  increased  attention  from  Army  leadership.  Thus 
far,  case  studies  have  been  completed  on  the  Long  Endur¬ 
ance  Multi-intelligence  Vehicle  (LEMV)  and  the  Armored 
Multi-Purpose  Vehicle  (AMPV).  Findings  from  these  efforts 
are  available  through  the  ALLP. 

In  addition  to  the  ALLP's  lesson-learned  search  page,  lessons 
are  disseminated  through  periodic  bulletins  and  article  pub¬ 
lications.  CAALL  distributes  new  lessons  and  other  informa¬ 
tion  on  the  current  activities  of  the  team  through  a  quarterly 
Acquisition  Lessons  Learned  Bulletin,  which  is  provided  to  all 
ALLP  users  and  Acquisition  Lessons  Learned  stakeholders. 
Current  readers  include  the  PEOs  and  other  acquisition  orga¬ 
nizations,  such  as  the  Army  Test  and  Evaluation  Command, 
Training  and  Doctrine  Command,  and  the  Defense  Acquisi¬ 
tion  University.  CAALL  also  regularly  provides,  to  the  Army 


AL&J  magazine,  articles  that  tie  lessons  learned  and  best 
practices  into  the  magazine's  current  theme.  In  an  effort  to 
push  lessons  to  the  project  offices,  CAALL  has  prepared  "Just 
In  Time"  lesson-learned  packages,  each  of  which  contains  a 
set  of  lessons  grouped  by  category  (such  as  contracting,  test 
and  evaluation,  systems  engineering,  etc.)  pertaining  to  a 
particular  phase  of  the  acquisition  life  cycle.  These  packages 
will  be  available  through  the  ALLP,  as  well  as  disseminated  to 
a  point  of  contact  at  each  PEO  so  that  PMs  and  their  staffs 
may  be  provided  with  relevant  lessons  learned  up  front  when 
entering  a  new  phase  of  the  acquisition  life  cycle.  These  pack¬ 
ages  will  facilitate  the  sharing  of  repeatable  good  practices 
and  knowledge  from  past  mistakes  with  other  project  offices 
so  they  may  benefit  from  the  knowledge  of  those  programs 
that  have  completed  that  acquisition  phase. 

Current  Acquisition  Lessons  Learned 

The  ALLP  houses  a  wide  range  of  acquisition  lessons  learned 
from  across  the  acquisition  life  cycle.  Lessons  pertain  to  a 
variety  of  topics,  such  as  program  management,  technology 
and  engineering,  contracting  and  financial  management.  High¬ 
lighted  below  are  a  few  lessons  currently  available  in  the  ALLP. 

Oversight  Review  and  Documentation 

One  of  the  largest  trends  within  the  current  collection  of 
Army  acquisition  lessons  learned  is  the  need  for  early  and 
efficient  milestone  and  documentation  preparation.  Pro¬ 
grams  have  repeatedly  cited  the  need  to  coordinate  with 
stakeholders  early,  utilize  documentation  Integrated  Prod¬ 
uct  Teams  and  tracking  tools,  and  know  what  is  required 
for  the  milestone  review.  One  program  reported  that  the 
PM  often  needs  to  allocate  critical  resources  to  produce 
and  staff,  or  obtain  waivers,  for  documents  that  are  not 
relevant  to  the  program.  The  program  had  to  meet  tradi¬ 
tional  documentation  requirements  for  its  milestone  re¬ 
view,  which  could  have  been  averted  if  Army  leadership 
would  allow  certain  requirements  (such  as  the  Corrosion 
Prevention  and  Control  Plan,  Program  Protection  Plan,  and 
Clinger-Cohen  Compliance)  to  be  declared  inapplicable  for 
programs  that  have  existing  materiel  solutions  and/or  that 
don't  have  certain  components  (such  as  electronics).  Prior 
to  the  milestone  review,  the  program  had  a  materiel  solution 
that  was  currently  in  use  in  the  field,  and  contracts  were 
already  In  place  and  producing  the  system  to  support  de¬ 
ploying  warfighters  at  a  rate  equivalent  to  Full  Rate  Produc¬ 
tion.  However,  the  program  still  had  to  complete  or  obtain 
waivers  for  statutory,  regulatory  and  policy  requirements 
to  achieve  the  milestone.  The  program  recommended 
that  PMs  seek  ASA(ALT)/Headquarters,  Department  of 
the  Army,  approval  for  the  Milestone  Decision  Authority 
to  declare  certain  documents  to  be  Inapplicable  for  cer¬ 
tain  programs  instead  of  requiring  a  waiver  or  streamlined 
version  of  the  document.  A  waiver  or  streamlined  version 
can  take  almost  as  much  time  to  prepare  and  staff  as  the 
traditional  document.  This  would  have  saved  the  program 
many  hours  that  could  be  used  In  other  initiatives  and  would 
have  allowed  the  milestone  to  be  executed  months  earlier. 
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Program  Management 

Another  Army  program  reported  that  PMs  need  to  be  pro¬ 
active  and  deliberate  in  initiating  and  establishing  an  Earned 
Value  Management  System  (EVMS)  for  production  activities 
at  arsenals  and  depots.  Proper  Earned  Value  Management 
metrics  were  not  established  at  the  beginning  of  the  program 
between  the  PM  and  the  arsenal.  The  product  office  did  not 
have  EVMS  or  a  similar  management  tool  in  place  to  estab¬ 
lish  a  planning  baseline  or  to  measure  cost  and  schedule  per¬ 
formance  over  time.  The  lack  of  these  tools  contributed  to 
what  became  a  more  than  $41  million  cost  overrun  and  a  10- 
month  schedule  slip.  The  arsenal  relies  on  a  logistics  system 
that  proved  inadequate  for  tracking  earned  value,  defining  the 
estimate  at  completion  and  managing  end-to-end  parts  ac¬ 
quisition,  production  and  costs.  The  system  would  not  allow 


consider  test  efficiencies  during  test-plan  development. 
This  includes  increasing  the  number  of  test  articles,  com¬ 
bining  different  types  of  tests,  using  test  data  from  similar 
programs  and  using  modeling  and  simulation.  One  program 
reported  that  using  similar  program-and-design-level  test 
data  can  reduce  the  number  of  required  tests  for  the  field¬ 
ing  of  a  system.  The  program  had  requirements  to  undergo 
some  very  expensive  tests  of  multiple  systems  and  subsys¬ 
tems.  The  program  initiated  a  study  to  evaluate  alterna¬ 
tive  solutions  that  would  satisfy  the  testing  requirements 
in  the  most  cost-effective  way.  For  this  study,  the  program 
reviewed  all  test  data  from  similar  systems  that  had  un¬ 
dergone  these  types  of  tests  so  the  program  could  predict 
through  test  knowledge  the  results  of  the  test.  These  pre¬ 
dictions  and  results  by  similarity  were  briefed  to  the  testing 


for  material  to  be  charged  to  a  program  until  it  is  used,  which 
can  be  several  months  after  it  has  been  purchased.  Since  the 
system  has  no  automatic  triggers  or  warnings,  the  arsenal 
would  be  able  to  continue  to  charge  against  a  Military  Interde¬ 
partmental  Purchase  Request  (MIPR)  even  if  funds  had  been 
exhausted.  Consequently,  the  program  provided  the  following 
recommendations  for  applying  Earned  Value  Management 
type  oversight: 

■  Conduct  Start  of  Work  Meetings  and  incorporate  EVMS 
up  front. 

■  Execute  disciplined  routine  Program  Management  Re¬ 
views  that  focus  on  cost,  schedule  and  performance. 

■  Utilize  Integrated  Master  Schedule/Integrated  Master 
Plan  tools  to  measure  program  performance. 

■  Update  cost  estimates  as  the  program  evolves  and  every 
time  the  scope  changes. 

■  Establish  a  measurable  baseline  at  the  outset  of  the  pro¬ 
gram. 

■  Provide  monthly  cost  reports  to  Life  Cycle  Management 
Command,  customers  and  installation  Commanders. 

■  Continuously  refine  metrics  and  reporting. 

Test  and  Evaluation 

Another  trend  that  has  begun  emerging  from  acquisition 
lessons  submitted  by  the  PEO  community  is  the  need  to 


board,  and  a  reduced  test  matrix  was  proposed  leveraging 
these  similar  program  and  past  design-level  test  results. 
The  final  test  matrix  required  only  33  percent  of  tests  origi¬ 
nally  planned/required,  and  the  tests  that  were  ultimately 
required  were  easy  to  fit  into  the  schedule  for  fielding  the 
system.  The  program  recommended  that  other  programs 
review  their  design  tests  and  similar  system  tests  to  le¬ 
verage  this  test  data  when  putting  together  the  required 
test  matrix  to  meet  materiel  release  requirements.  The 
benefits  of  understanding  the  testing  and  ways  to  predict 
results  based  on  similarity  or  design-level  tests  saved  the 
program  $1.86  million  in  hardware  and  test-range  costs.  In 
addition,  the  program  benefited  from  a  shortened  schedule 
and  reduced  travel  costs  to  meet  the  schedule  required  for 
materiel  release  of  the  system. 

The  ALLP  continues  to  rely  on  valuable  submissions  from 
across  the  Army  acquisition  enterprise.  These  and  other  ac¬ 
quisition  lessons  learned  may  be  accessed  through  the  portal 
at  https://allp.amsaa.army.mil/.  The  ALLP  is  open  to  all  DoD 
military  and  civilians,  and  AMSAA  welcomes  you  to  contrib¬ 
ute  to  this  valuable  mission  by  sharing  your  knowledge  and 
experiences,  as  well  as  leveraging  those  lessons  currently  in 
the  portal.  ^ 


The  author  can  be  contacted  at  jni.l.iracki.civ@man.mn. 
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